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The  Effects  of  Workload  and  Decision  Support  Automation  on 
En  Route  R-Side  and  D-Side  Communication  Exchanges 


Federal  Aviation  Administration  (FAA)  forecasts 
indicate  that  the  number  of  passengers  carried  on 
commercial  aircraft  will  double  over  the  next  1 5  years 
reaching  one  billion  by  2015  (FAA  Plan  to  Modern¬ 
ize,  1998).  To  handle  the  projected  increase  in  air 
traffic,  the  FAA  is  introducing  new  technology  to  aid 
air  traffic  control  specialists  (ATCSs)  in  their  tactical 
and  strategic  decision  making.  Research  previously 
focused  on  the  effects  of  these  decision  aids  on  pilot- 
to-controller  communication;  however,  relatively  little 
research  is  available  concerning  the  impact  of  decision 
aids  on  controller-to-controller  (CTC)  communica¬ 
tion  (Kanki  &  Pinzo,  1996).  In  this  study,  we  exam¬ 
ined  the  effects  of  aircraft  density  and  automated 
decision  aids  on  communication  exchanges  between 
en  route  air  traffic  control  teams. 

The  National  Civil  Aviation  Review  Commission 
states  that  the  expected  growth  in  aviation  cannot  be 
safely  accommodated  without  significant  break¬ 
throughs  in  air  traffic  modernization.  The  commis¬ 
sion  also  cites  air  traffic  communications  as  critical 
components  requiring  modernization  in  the  aviation 
system  (Aviation  Financing,  Air  Traffic  Control, 
1999).  Several  emerging  air  traffic  control  systems, 
including  the  Voice  Switching  and  Control  Sys¬ 
tem  (VSCS),  Display  System  Replacement  (DSR), 
and  the  User  Request  Evaluation  Tool  (URET)  are  in 
differing  stages  of  implementation.  Researchers  at  the 
Civil  Aerospace  Institute  (CAMI)  and  the  William  J. 
Hughes  Technical  Center  (WJHTC)  are  joindy  con¬ 
ducting  investigations  to  assess  the  impact  of  moderniz¬ 
ing  ATC  communications.  This  research  considers  both 
hardware  and  software  aspects  of  ATC  modernization. 

The  current  joint  program  of  research  focuses  on 
CTC  communications  as  they  relate  to  the  coordina¬ 
tion  of  job-related  tasks.  The  present  study  examines 
communication  exchanges  between  En  route  Sector 
Teams  (ESTs),  also  known  as  R-side  and  D-side  air 
traffic  controller  teams,  performing  under  varying 
levels  of  workload  and  using  various  decision  support 
tools  (DSTs).  ESTs  are  composed  of  two  individuals, 
a  radar  controller  (R-side)  and  a  data  controller  (D- 
side).  The  radar  controller’s  responsibilities  include 
monitoring  aircraft,  maintaining  aircraft  separation, 
and  communicating  via  radio  with  the  aircraft  in  their 


airspace  (i.e.,  sector).  The  data  controller’s  duties 
include  maintaining  flight  progress  strips,  entering 
National  Airspace  System  (NAS)  data,  coordinating 
sector  activities,  and  conducting  intra-facility  and 
inter-facility  communication.  The  above  functions 
are  coordinated  through  CTC  communication. 

Operational  Error  Data  Base 

Tracking  the  number  of  operational  errors  is  one 
way  of  evaluating  the  quality  of  intra-EST  coordina¬ 
tion.  Broadly  speaking,  an  operational  error  occurs 
when  a  deviation  from  the  standard  operational  pro¬ 
cedures  (SOPs)  leads  to  a  loss  of  aircraft  separation. 
For  example,  in  the  en  route  environment,  aircraft  at 
the  same  altitude  must  be  separated  by  at  least  five 
horizontal  miles  (FAA  Order,  2000).  Any  time  a 
verified  loss  of  separation  occurs,  an  operational  error 
(OE)  is  recorded  in  the  FAA’s  Operational  Error 
database,  and  an  investigation  is  conducted  to  deter¬ 
mine  the  factors  contributing  to  the  loss  of  separation. 

During  an  operational  error  investigation,  an  op¬ 
erational  error  reporting  form  is  used  to  help  deter¬ 
mine  the  factors  contributing  to  the  error.  As  shown 
in  Table  1 ,  the  reporting  form  is  divided  into  five  error 
categories.  These  include  errors  associated  with:  (1) 
Data  posting,  (2)  Radar  display,  (3)  Communication 
(primarily  pilot  to  controller),  (4)  Coordination,  and 
(5)  Position  relief  briefings. DW:  Each  error  category 
is  further  divided  into  varying  levels  of  specificity. 

Of  interest  in  this  report  are  the  “intra-position” 
errors  associated  with  “Coordination.”  In  the  en  route 
environment,  intra-position  coordination  primarily 
refers  to  problems  with  EST  coordination.  Since 
communication  is  the  means  by  which  the  R-side  and 
D-side  coordinate  their  respective  tasks,  errors  in 
coordination  include  errors  in  CTC  communica¬ 
tions.  Table  2  shows  the  number  and  percentage  of 
coordination  related  OEs  recorded  in  the  FAA’s  OE 
database.  For  the  period  from  January  1998  through 
October  2000  approximately  126  out  of  505  (25%) 
OEs  were  associated  with  problems  of  coordination. 
Of  the  126  coordination  errors  (CEs)  45  (36%)  oc¬ 
curred  in  the  en  route  environment.  A  breakdown  in 
intra-position  coordination  accounted  for  13  out  of 
45  (30%)  en  route  coordination  errors  (ECEs). 
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Table  1. 

Structure  of  FAA  ATC  Operational  Error  Reporting  Form 


1.  Data  Posting 

a.  Computer  entry 

(1)  Incorrect  input 

(2)  Incorrect  update 

(3)  Premature  termination  of  data 

(4)  Other 

b.  Flight  progress  strip 

(1)  Not  prepared 

(2)  Not  updated 

(3)  Posted  incorrectly 

(4)  Reposted  incorrectly 

(5)  Updated  incorrectly 

(6)  Sequenced  incorrectly 

(7)  Resequenced  incorrectly 

(8)  Interpreted  incorrectly 

(9)  Premature  removal 

(10)  Other 

2.  Radar  display 

a.  Misidentification 

( 1 )  Overlapping  data  blocks 

(2)  Acceptance  of  incomplete  or 
difficult  to  correlated  position  info. 

(3)  Improper  use  of  identifying  turn. 

(4)  Failure  to  reidentify  aircraft  when 
accepted  target  identity  becomes 
questionable. 

(5)  Failure  to  confirm  aircraft  identity 
after  accepting  radar  handoff 

(6)  Other 

b.  Inappropriate  use  of  displayed  data 

(1)  Conflict  alert 

(2)  Quick  look 

(3)  ModeC 

(4)  MSAW/EMSAW 

(5)  Other 

3.  Communication  error 

a.  Phraseology 

b.  Transposition 


c.  Misunderstanding 

d.  Readback 

(1)  Altitude 

(2)  Clearance 

(3)  Identification 

e.  Acknowledgment 
f  Other 

4.  Coordination 

a.  Area  of  incident 

(1)  Inter-position 

(2)  Intra-position 

(3)  Inter-sector 

(4)  Inter-facility 

b.  An  aircraft  penetrated  designated 
airspace  of  another  position  of  operation 
or  facility  without  prior  approval. 

c.  Coordination  was  effected  and 
controller(s)  did  not  utilize  information 
exchange. 

(1)  Aircraft  identification 

(2)  Altitudes/Flight  level 

(3)  Route  of  flight 

(4)  Clearance  limit 

(5)  Speeds 

(6)  APREQS 

(7)  Special  instructions 

(8)  Other 

5.  Position  relief  briefing  deficiencies  noted 

a.  Employee  did  not  used  position  relief 
Checklist 

b.  Employee  being  relieved  gave 
incomplete  briefing 

c.  Relieving  employee  did  not  make  use 
of  pertinent  data  exchanged  at  briefing 

d.  Other 
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Based  on  the  results  of  Table  2,  it  may  appear  that 
intra-position  coordination  problems  are  not  a  major 
factor  in  the  creation  of  operational  errors.  However, 
it  should  be  noted  that  operational  error  investiga¬ 
tions  occur  after  the  fact.  Since  intra-EST  communi¬ 
cations  are  not  recorded  on  tape,  an  account  does  not 
exist  of  the  actual  communication  exchanges  between 
R-side  and  D-side  positions.  Controllers  must  rely  on 
their  memory  to  reconstruct  both  the  content  and 
timing  of  the  communication  exchange  being  investi¬ 
gated.  Thus,  intra-EST  communications  may  be  a 
factor  in  the  creation  of  OEs,  but  the  current  report¬ 
ing  system  is  not  sensitive  enough  to  capture  the 
details.  Additionally  not  all  intra-position  coordina¬ 
tion  problems  produce  an  operational  error.  That  is, 
a  breakdown  in  coordination  may  occur  but  the 
problem  is  resolved  in  time  to  prevent  an  operational 
error.  Although  there  may  be  no  loss  of  aircraft  sepa¬ 
ration,  from  a  safety  perspective,  even  a  potential  loss 
of  separation  is  a  matter  of  concern.  A  research  initia¬ 
tive  revising  the  OE  reporting  form  to  address  these 
and  other  human  factors  related  to  the  systematic 
analysis  of  OEs  is  underway  at  CAMI  (Scarborough  & 
Pounds,  2001). 

Even  if  intra-EST  communications  prove  not  to  be 
a  major  factor  in  the  development  of  OEs,  the  need  to 
conduct  research  is  not  diminished.  Empirical  knowl¬ 
edge  of  the  daily  task-related  communication  ex¬ 
changes  between  R-side  and  D-side  controllers  is 
limited.  Considering  that  the  new  DSTs  are  designed 
to  enhance  the  tactical  and  strategic  decision  making 
capabilities  of  ESTs,  further  intra-EST  communica¬ 
tion  research  is  essential.  What  effect  might  these  new 
technologies  have  on  existing  patterns  of  intra-EST 
communications?  Without  an  understanding  of  the 
current  baseline  patterns  of  intra-EST  communications, 
there  is  no  empirical  way  to  answer  this  question. 


Field  Study 

Given  the  lack  of  understanding  of  intra-EST  com¬ 
munications,  Peterson,  Bailey  &  Willems  (in  press) 
conducted  a  field  study  to  determine  the  kinds  and 
frequency  of  intra-EST  communication  exchanges 
associated  with  routine  ATC  performance.  The  field 
study  was  conducted  over  a  three-day  period  at  one  of 
the  FAA’s  en  route  centers.  Trained  ATC  specialists 
using  the  FAA’s  Controller-to-Controller  Communi¬ 
cation/Coordination  Taxonomy  (C^T)  recorded  a 
total  of  24  hours  of  intra-EST  communication.  Cod¬ 
ing  occurred  in  30-45  minute  blocks  of  time  between 
the  hours  of  0700  and  1900.  Coders  chose  to  observe 
the  most  active  sectors  to  ensure  a  sufficient  number 
of  coding  events.  Thus,  rather  than  obtaining  a  statis¬ 
tical  sample,  coders  chose  a  sample  of  convenience. 

The  G^T  has  three  communication  categories:  the 
topic  of  communication,  the  grammatical  form  of 
communication  (e.g.  question,  answer),  and  the  mode 
of  communication  (e.g.  verbal,  nonverbal).  Thus  the 
C^T  captures  the  “what”  (topic)  and  “how”  (form  and 
mode)  of  communication.  Tables  3-5  respectively 
describe  the  G^T  sub  levels  within  each  of  the  three 
categories.  For  further  information  on  the  develop¬ 
ment  and  operational  validation  of  the  G^T,  the 
reader  is  referred  to  Peterson,  Bailey  and  Willems  (in 
press) . 

A  frequency  analysis  of  the  coding  results  indicated 
the  following  observations.  The  topic  of  most  intra- 
EST  communications  was  related  to  aircraft  traffic 
(40%)  and  route  of  flight  (15%),  with  the  least  com¬ 
munications  involving  inter-sector  coordination  ap¬ 
provals  (1%).  R-side  and  D-side  controllers 
demonstrated  no  statistical  differences  in  the  topic  of 
their  communication.  However,  they  differed  in  the 
grammatical  form  of  their  communication.  Whereas 
the  D-side  had  a  higher  percentage  of  statements  and 


Table  2. 

Breakdown  of  En  route  Coordination  Errors  (1998-2000) 


Year 

All 

All  ATC 
Coordination 

All  En  Route 
Coordination 

Inter-Position 

Intra-Position 

Inter- Sector 

Inter-Facility 

ATC 

Errors  (CEs) 

Errors  (ECEs) 

ECE 

ECE 

ECE 

ECE 

OEs 

#  %OE 

#  %CE 

#  %ECE 

#  %ECE 

#  %ECE 

#  %ECE 

1998 

517 

154 

30% 

47 

31% 

24 

51% 

11 

23% 

0 

0 

12 

26% 

1999 

627 

167 

27% 

59 

35% 

30 

51% 

21 

35% 

0 

0 

8 

14% 

2000* 

371 

58 

16% 

29 

50% 

19 

66% 

8 

27% 

0 

0 

2 

7% 

Average 

505 

126 

25% 

45 

36% 

24 

53% 

13 

30% 

0 

0 

7 

16% 

*  Through  October  1, 2000 
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Table  3. 

Controller-to-Controller  Coordination  Communications  Taxonomy  (C^'T): 
Coordination-Communication  Topic 


ATC  Coordination- 
Communication 
Topic 

Detinitions 

Approval 

Communications  about  intersector  control/approval  requests  (“Get  me  control  for  descent 
on  that  aircraft.”  “APREQ  N1234  climbing  to  FL330.”). 

Handoff 

Communications  relating  to  the  transfer  of  radar  identification  of  a  particular  aircraft 
(“Handoff  N 1234.”  “Did  you  handoff  N 1234?”). 

Point  Out 

Communications  relating  to  the  transfer  of  radar  identification  of  a  particular  aircraft  when 
radio  communications  will  be  retained  (“Point  out  N1234  to  22.”). 

Traffic 

Communications  about  a  traffic  situation  involving  a  specific  aircraft.  Includes  conflict, 
spacing,  other  protected  air  space  or  terrain  and  the  resolution  of  that  situation  (“Are  you 
watching  that  aircraft?”). 

Altitude 

Communications  about  altitude  not  in  relation  to  traffic  (“N1234  is  requesting  flight  level 
220.”). 

Route 

Communications  regarding  headings  and/or  amendments  to  route,  not  in  relation  to  traffic 
situations  (“N1234  is  on  a  330  heading.”  “Next  sector,  27,  wants  N1234  over  WEVER.”). 

Speed 

Communications  about  speed  not  in  relation  to  traffic  situations  (‘These  three  aircraft  are 
slowed  to  250  knots.”). 

Weather 

Communications  about  weather  display  or  weather  updates  (Often  communicated 
nonverbally  by  passing  written  information:  “Sector  22  says  continuous  moderate 
turbulence  above  FL290.”). 

Frequency 

Communications  about  an  aircraft’s  radio  communications  transfer  or  frequency 
assignment  (“Have  you  switched  N 1234  yet?”  ‘Tell  them  to  switch  to  N1234.”). 

Flow  Messages 

Communications  about  traffic  flow  restrictions  not  referring  to  a  specific  aircraft  (‘The 
next  sector  is  requesting  25  miles  in  trail.”)  (due  to  radar  outage). 

Flight  Strips 

Communications  about  flight  progress  strips  (“Where  is  that  strip?”)  Often  communicated 
nonverbally. 

Equipment 

Communications  about  any  ATC  hardware  (The  radar  is  out  of  service.”). 

Aircraft  BO 

Communications  involving  identifying  a  specific  aircraft  (Who  was  that  who  called?” 

‘That  was  N1234  who  called.”). 

Table  4. 

Controller-to-Controller  Coordination  Communications  Taxonomy  (C^'T):  Coordination-Communication 
Grammatical  Form. 


ATC  Coordination- 
Communication 
Grammatical  Form 

Definitions 

Question 

A  direct  inquiry  about  the  state  or  status  of  sector  events. 

Answer 

A  response  to  a  direct  or  implied  question 

Statement 

Providing  information,  without  being  asked,  about  the  state  or  status  of  sector  events. 

Command 

A  direct  order  to  perform  a  specific  act 
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Table  5. 

Controller-to-Controller  Coordination  Communications  Taxonomy  (C'h’):  Coordination-Communication 
Expression 


ATC  Coordination- 
Communication  Form 


Detinitions 


Verbal 


Use  of  voice  only  communication. 


Nonverbal 

Mixed 

Electronic 


Use  of  only  body  movement  communication. 

Communication  that  contains  both  a  verbal  and  non  verbal  component. 
Not  used.  Communication  that  is  electronically  transferred. 


observations  (56%  vs.  30%),  the  R-side  had  a  higher 
percentage  of  answers  (43%  vs.  35%).  From  this,  it 
appeared  that,  compared  with  the  R-side,  the  D-side 
controller  was  the  initiator  of  more  communication. 
Finally,  there  was  no  intra-EST  difference  in  the 
method  of  communication.  The  most  frequent  method 
of  intra-EST  communication  was  verbal  only  (70%). 
The  remaining  30%  of  communications  contained  a 
mixture  of  verbal  and  non-verbal  expressions.  This 
latter  finding  suggested  that  any  changes  affecting  the 
line  of  sight  between  R-side  and  D-side  positions 
could  disrupt  the  adaptive  use  of  intra-EST  nonverbal 
communication . 

Peterson  et  al.  (in  press)  concluded  their  report  by 
highlighting  the  need  to  conduct  controlled  experi¬ 
ments  to  determine  how  changes  in  workload  and 
technology  affects  intra-EST  communications.  Al¬ 
though  field  studies  preserve  operational  realism,  with 
a  complex  task  such  as  air  traffic  control,  many  vari¬ 
ables  come  into  play,  making  it  difficult  to  determine 
which  factors  affect  performance.  For  example,  one 
might  wish  to  know  the  effects  of  more  or  less  CTC 
communication  on  sector  safety  and/or  efficiency. 
Flowever,  CTC  communication  is  the  result  of  the 
interaction  between  the  EST  members  and  the  envi¬ 
ronment  (sector  complexity  and  traffic  volume)  in 
which  they  work.  Within  a  laboratory  environment, 
researchers  have  greater  control  over  the  environmen¬ 
tal  setting  (e.g.,  equipment,  sector  complexity,  traffic 
volume,  work  duration)  and,  thus,  can  better  under¬ 
stand  the  effects  of  the  experimental  manipulation. 

Collaborative  Research 

Although  researchers  have  greater  control  in  a  labo¬ 
ratory  environment,  sometimes  the  experimental  en¬ 
vironment  does  not  adequately  represent  operational 
reality.  When  this  happens,  it  is  difficult  to  generalize 


from  experimental  outcomes  to  the  operational  envi¬ 
ronment.  One  way  of  addressing  the  artificial  nature 
of  a  laboratory  setting  is  to  ensure  that  the  equipment, 
participants,  and  stimulus  material  reflect  real  world 
conditions.  This  requires  significant  financial  re¬ 
sources.  The  hardware  and  software  support  necessary 
to  simulate  a  high  fidelity  en  route  air  traffic  control 
environment  costs  in  excess  of  $500,000.  In  addition, 
the  cost  for  each  ATC  participant  is  $2500/week. 

Given  that  research  in  the  area  of  intra-EST  com¬ 
munication  is  at  the  descriptive  stage  of  empirical 
research,  large  expenditures  devoted  solely  to  describ¬ 
ing  CTC  communications  are  difficult  to  justify.  To 
help  reduce  the  cost  of  research,  researchers  from 
CAMI  and  the  WJHTC  developed  a  collaborative 
initiative  whereby  intra-EST  communication  data  are 
collected  on  all  experiments  related  to  the  WJFITC’s 
Decision  Support  Automation  Research  (DSAR)  pro¬ 
gram  (Willems,  1999).  The  research  objective  of  DSAR 
is  to  evaluate  ATC  human  performance  under  varying 
levels  of  workload  and  DSTs.  Thus,  DSAR  experi¬ 
ments  provide  an  ideal  setting  for  the  study  of  intra- 
EST  communications. 

Research  Hypotheses 

During  the  descriptive  phase  of  research,  objectives 
focus  more  on  systematically  collecting  and  organiz¬ 
ing  data  than  it  does  on  hypothesis  testing.  However, 
a  body  of  literature  exists  on  controller- to-pilot  com¬ 
munication.  Using  this  literature  as  a  starting  point, 
some  initial  hypotheses  can  be  generated  about  the 
effects  of  workload  and  technology  on  communications. 

Workload 

A  consistent  finding  in  controller-to-pilot  voice 
communications  research  is  that  workload  affects  the 
quality  and  quantity  of  communication  exchanges 
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(e.g.,  Prinzo  &  Britton,  1993).  In  this  literature, 
workload  is  primarily  measured  by  the  number  of 
aircraft  at  a  given  time  (i.e.,  aircraft  density)  under  the 
control  of  a  R-side  ATCS.  As  aircraft  density  in- 
creases,  there  is  a  corresponding  trend  toward  an 
increasing  number  of  communication  errors  (Morrison 
&  Wright,  1989;  Morrow,  Lee,  &  Rodvold,  1990). 
Research  suggests  that,  as  ATCS  and  pilots  become 
overburdened  the  clarity  of  their  communications 
(e.g.,  incomplete  phraseology,  mispronunciation,  and 
rapid  speech)  begin  to  suffer.  This,  in  turn,  places 
ATCS  and  pilots  at  a  higher  risk  of  committing 
readback/hearback  errors  (Morrison  &  Wright,  1989). 
Standard  ATC  protocol  requires  pilots  and  ATCS  to 
repeat  what  they  heard.  The  sender  then  knows  that 
the  message  was  accurately  received.  A  readback/ 
hearback  error  occurs  when  an  incorrect  pilot  or 
ATCS  readback  of  information  goes  uncorrected. 

In  addition  to  communication  errors,  changes  in 
workload  also  affect  the  kinds  of  communication 
exchanges  that  occur.  For  example,  researchers  at 
Human  Technology  Incorporated  (1991)  examined 
the  effects  that  ATC  communications  had  on  system 
performance.  They  found  that  high  performance  teams 
issued  more  communication  reports  to  pilots  than  did 
lower  performance  teams.  Furthermore,  compared  to 
low  performance  teams,  high  performance  teams  is¬ 
sued  shorter  messages  as  a  means  of  insuring  accuracy. 
Ratings  of  team  performance  were  based  on  over-the- 
shoulder  ratings  conducted  by  ATC  subject  matter 
experts.  However,  it  remains  to  be  seen  whether  these 
results  will  generalize  to  the  broader  ATC  population 
or  are  specific  to  a  given  experimental  manipulation. 

Automation 

Technology  is  sometimes  considered  to  be  the 
answer  for  life’s  problems.  However,  technology  itself 
can  become  a  burden.  In  studies  of  the  use  of  automa¬ 
tion  in  the  cockpit.  Hart  and  Sheridan  (1984)  found 
that  pilot  workload  shifted  from  a  physical  burden  to 
a  more  cognitive  one.  This  was  evident  when  the 
automation  required  pilots  to  serve  as  monitors  of  a 
system  such  as  monitoring  cockpit  flight  management 
system  displays  (Sarter  &  Woods,  1994).  In  studies  of 
the  use  of  automation  in  commercial  airlines,  Wiener 
(1985)  noted  that  pilots  found  the  additional  task  of 
monitoring  to  be  troublesome  especially  during  high 
workload.  Not  only  did  monitoring  add  to  the  fatigue 
but  there  were  times  when  the  automation  needed  to 
be  turned  off  or  ignored  during  critical  portions  of  a 
flight  (Wiener,  1988). 


In  addition  to  the  added  burden  that  some  tech¬ 
nologies  create,  operators  may  not  use  the  technology 
as  designed  because  they  simply  do  not  trust  it  (Riley, 
1994).  Technology  promises  much,  but  the  reality  is 
that  it  sometimes  falls  short  on  delivery.  For  example, 
as  part  of  its  ATC  modernization  program,  the  FAA 
developed  new  data  display  terminals  and  keyboard 
configurations.  It  was  expected  that  the  new  system 
would  help  to  improve  system  capacity.  However, 
when  the  equipment  was  tested  in  a  field  setting,  users 
found  that  they  were  having  problems  making  the 
transition  to  the  new  system  (Allendoerfer,  Galushka, 
&  Mogford,  2000).  Compared  with  performance 
under  the  old  system,  ATCSs  were  slower  and  tended 
to  make  more  data  entry  errors.  Rather  than  viewing 
this  as  simply  a  matter  of  adjustment,  users  attributed 
the  problems  that  they  were  having  to  poor  system 
design..  Although  some  sort  of  an  adjustment  period 
was  expected,  problems  such  as  these  emphasize  the 
need  to  switch  from  technology  centered  approaches 
to  more  human  centered  approaches  to  moderniza¬ 
tion.  Whereas  the  former  requires  the  human  to  adapt 
to  the  technology,  the  latter  requires  the  technology  to 
adapt  to  the  needs  of  the  user  (Billings,  1988). 

Although  the  use  of  technology  is  related  to 
workload,  its  effects  on  communication  are  unclear. 
From  the  concept  of  monitoring,  an  operator  has  no 
need  to  communicate  unless  an  event  occurs  requiring 
the  actions  of  another.  As  far  as  whether  a  given 
technology  is  used  or  trusted,  one  might  conjecture 
that  communications  would  increase  during  the  tran¬ 
sition  period  of  adjusting  to  the  new  technology.  This 
would  be  true  if  there  were  problems  with  the  technol¬ 
ogy  and  team  members  had  to  determine  the  level  of 
trust  or  confidence  in  the  system.  However,  once  the 
transition  period  is  completed  (and  assuming  that  the 
system  is  reliable),  one  would  expect  communications 
to  return  to  a  previous  baseline. 

Considering  the  above  discussion,  the  following 
hypotheses  were  derived. 

Hypothesis  1:  More  intra-EST  communication 
exchanges  will  occur  under  high  workload  condi¬ 
tions,  as  compared  with  low  workload  conditions. 

Hypothesis  2:  More  intra-EST  communication 
exchanges  will  occur  when  using  DSTs,  as  compared 
with  not  using  DSTs.  This  hypothesis  is  based  on  the 
assumption  that  there  will  be  insufficient  training 
time  for  participants  to  feel  fully  comfortable  with  the 
new  system  procedures. 


6 


Study  Limitations 

One  of  the  inherent  difficulties  with  conducting 
applied  research  is  working  within  limitations.  Two 
limitations  that  affect  this  current  research  are  a  lack 
of  statistical  power  and  financial  resources.  Statistical 
power  refers  to  the  minimal  sample  size  necessary  for 
a  given  effect  to  reach  statistical  significance.  As  cell 
sizes  fall  below  this  minimum,  a  the  ability  to  ad¬ 
equately  test  hypotheses  is  reduced. 

Prior  to  conducting  the  experiments,  researchers 
conducted  a  power  analysis  using  software  from  the 
Methodologist's  Toolchest  (Brent  &  Thompson, 
1996).  The  procedure  is  derived  from  Kraemer  and 
Theimann  (1987)  for  related  samples  balanced  de¬ 
signs  on  pair-wise  comparisons  with  a  two-tailed  test 
of  significance.  Assuming  a  mean  difference  of  2,  an 
average  standard  deviation  of  10,  an  alpha  of  .10,  and 
a  beta  of  .50,  the  results  produced  a  minimum  sample 
size  of  18  cases  (or  ESTs)  or  a  total  of  36  ATCSs. 
However,  in  this  study,  the  available  financial  re¬ 
sources  could  only  cover  the  cost  of  8  ESTs  or  a  total 
of  16  ATCSs.  As  is  often  the  case  in  applied  settings, 
economic  realities  become  the  limiting  factor  in  the 
quality  of  the  studies  conducted. 

Given  the  above  limitations,  one  might  ask,  is  there 
any  real  value  in  studying  intra-EST  communications 
in  a  laboratory  setting?  The  answer,  of  course,  de¬ 
pends  on  the  value  that  is  placed  on  inferential  statis¬ 
tics.  There  exists  the  concern  among  part  of  the 
scientific  community  that,  unless  results  achieve  sta¬ 
tistical  significance,  they  are  not  meaningful  or  are 
just  artifacts  of  sampling  error  (e.g..  Branch,  1999). 
What  gets  lost  is  that  statistical  significance  does  not 
guarantee  that  the  results  will  generalize  to  the  broader 
population.  Generalization  depends  on  how  the  sample 
was  selected  to  ensure  representation  (i.e.  generaliza¬ 
tion  back  to  the  population).  Continuing  with  the 
preceding  example,  let  us  assume  that  we  have  a  popula¬ 
tion  of  10,000  enroute  ATCSs  or  5,000  ESTs.  Accord¬ 
ing  to  sampling  theory,  we  would  have  to  draw  a  random 
sample  of 357  ESTs  to  ensure  that  our  sample  represents 
the  enroute  population.  Sample  computations  were 
made  using  the  website  calculator  at 
http://ebook.stat.ucla.edu/calculators/sampsize.phtml. 

Note  that  an  additional  321  ATCSs  are  needed  to 
ensure  representation  as  compared  to  statistical  sig¬ 
nificance.  If  achieving  statistical  power  is  cost  pro¬ 
hibitive,  representation  is  even  more  so. 

One  way  of  dealing  with  the  problem  of  inadequate 
sample  size  is  evaluating  intra-EST  communication 
patterns  across  numerous  studies.  Over  time,  a  com¬ 
munication  profile  develops  based  on  known  contex¬ 
tual  factors  (i.e.,  experimental  conditions).  To  the 


extent  that  similarities  in  the  communication  profile 
are  observed,  this  would  suggest  a  phenomena  that  is 
robust  and  thus  indicative  of  the  universal  nature  of 
R-side  and  D-side  communication.  However,  to  the 
extent  that  the  communication  profiles  are  different, 
this  would  suggest  that  situation-specific  conditions 
dictated  the  nature  of  intra-EST  communications. 
Thus,  regardless  of  the  outcome,  the  systematic  col¬ 
lection  of  data  has  the  potential  to  provide  insight  into 
the  nature  of  intra-EST  communications  and  to  pro¬ 
vide  guidelines  for  conducting  field  studies. 

METHOD 

The  research  described  in  this  report  is  a  subset  of 
a  broader  experiment  associated  with  the  WJHTC’s 
DSAR  program.  Only  the  methods  directly  related  to 
the  assessment  of  CTC  communications  are  reported 
in  this  study.  For  additional  information  on  the 
experimental  design,  see  Willems  (1999). 

Participants 

Eight  2-person  teams,  consisting  of  certified  ATCSs 
from  an  en  route  center  participated  in  a  two-week 
DSAR  experiment.  Participants  were  paid  their  regu¬ 
lar  salary  and  were  on  government  per  diem  through¬ 
out  the  duration  of  the  experiment.  Following  training, 
participants  were  randomly  assigned  to  an  experimental 
condition. 

Equipment 

The  equipment  used  in  this  experiment  was  function¬ 
ally  equivalent  to  the  workstations  used  by  the  R-side 
and  D-side  ATCSs  in  an  en  route  center.  An  ATC  high- 
fidelity  simulator  was  used  to  model  the  airspace  used  in 
this  study.  The  experimental  environment  included  full 
DSR  workstations  with  full  operational  functionality.  A 
2,000  by  2,000  pixel,  29”  video  display  unit  represented 
the  DSR  radarscope.  Also  included  were  a  DSR  flight 
strip  bay,  an  en  route  keyboard  and  trackball,  and  a  DST 
terminal  that  included  both  a  conflict  avoidance  tool 
(CAT)  and  a  flight  path  planning  tool  (FPPT).  Prior  to 
the  experiment,  none  of  the  participants  had  used  the 
CAT  or  FPPT. 

The  CAT  used  the  trajectory  of  aircraft  to  predict 
potential  loss  of  aircraft  separation.  For  example,  all 
things  being  constant,  the  CAT  determined  whether 
the  trajectory  of  two  aircraft  would  intersect  any  time 
within  a  20-minute  period.  Conflict  information  was 
presented  in  several  display  windows  that  depicted 
various  flight  data  and  conflict  information.  One 
display  window  presented  a  list  of  all  aircraft  inbound 
to  the  sector.  The  list  showed  who  was  the  controlling 
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sector  of  a  particular  aircraft;  flight  data  such  as  flight 
route,  aircraft  type,  speed,  altitude  and  beacon  code, 
and  conflict  indication.  The  conflict  indication  dis¬ 
played  red  for  predicted  violations  of  less  than  5 
nautical  miles  and  displayed  yellow  when  less  than  12 
nautical  miles,  but  more  than  5  nautical  miles.  A 
graphic-plan-display  window  graphically  depicted 
aircraft  and  resembled  the  DSR  display. 

By  a  D-side  entering  a  revised  altitude  or  route 
change,  the  FPPT  enabled  the  EST  to  determine  the 
best  flight  plan  for  resolving  a  potential  intra-  and/or 
inter-sector  conflict.  Thus,  instead  of  resolving  con¬ 
flicts  as  tactical  decisions,  the  FPPT  enabled  ATCSs 
to  choose  a  strategic  resolution  to  a  given  problem. 
Within  an  EST,  information  from  the  DSTs  was  pro¬ 
cessed  by  the  D-side  and  communicated  to  the  R-side. 

Stimulus  Material 

An  ATC  supervisor  on  detail  to  the  WJHTC  devel¬ 
oped  air  traffic  control  scenarios  for  use  in  the  training 
and  experimental  conditions.  Each  scenario  was  45 
minutes  long.  For  the  training  condition,  six  sce¬ 
narios  placed  participants  under  a  moderate  workload 
defined  as  the  amount  of  air  traffic  that  could  be 
comfortably  handled  by  a  R-side/D-side  team,  as 
perceived  by  a  typical  ATC  supervisor.  For  the  experi¬ 
mental  conditions,  six  low-and  seven  high-workload 
scenarios  were  developed.  The  low-workload  sce¬ 
narios  were  defined  as  the  least  amount  of  air  traffic  in 
which  a  typical  ATC  supervisor  added  a  D-side  to 
assist  the  R-side  controller.  The  standard  for  develop¬ 
ing  high  workload  scenarios  was  the  greatest  amount 
of  air  traffic  that  a  typical  ATC  supervisor  allowed  a 
R-side  /  D-side  team  to  manage. 

Measures 

All  task-related  R-side/D-side  communications  were 
videotaped  and  coded  by  an  ATC  subject  matter  expert 
(SME)  using  a  computerized  version  of  the  C^T.  The 
SME  was  a  retired  AT CS  who  had  spent  the  last  five  years 
providing  contract  support  in  the  area  of  ATC  commu¬ 
nications  research.  This  involved  providing  subject  mat¬ 
ter  expertise  in  the  development  of  coding  taxonomies 
for  both  controller-  to-pilot  communications  and  con- 
troller-to-controller  communications. 

Training 

Prior  to  participating  in  the  experiment,  participants 
received  four  days  of  training  on  the  air  space,  scenario 
flow  and  traffic  type,  DSR  workstations,  and  DST 
equipment.  After  completing  the  familiarization  phase 
of  training,  participants  performed  ATC  tasks  in  response 
to  six  45-minute  air  traffic  scenarios.  Each  scenario  was 


calibrated  by  a  SME  to  represent  a  moderate  amount  of 
workload.  The  decision  aids  used  during  training  fol¬ 
lowed  the  experimental  design. 

Design  and  Procedures 

Communication  exchanges  of  team  members  were 
assessed  within  a  2  (workload)  X  3  (type  of  DST) 
doubly  repeated  measures  design.  The  two  levels  of 
workload  were  low  and  high,  as  previously  described 
in  the  Stimulus  Material  section.  DST  levels  were:  (1) 
Tech  1- only  paper  flight  strips,  (2)  Tech  2  -  electronic 
flight  strips  and  a  CAT,  and  (3)  Tech  3-  electronic 
flight  strips,  CAT,  and  a  FPPT. 

RESULTS 

Field  Comparison 

Prior  to  hypothesis  testing,  the  frequency  data 
(collapsed  across  all  trials)  were  analyzed  by  topic, 
grammatical  form,  and  communication  mode  so  that 
a  direct  comparison  could  be  made  with  the  field 
study  conducted  by  Peterson,  et  al  (in  press).  The 
purpose  of  this  comparison  was  to  determine  if  differ¬ 
ences  existed  in  the  patterns  of  communication  oper¬ 
ating  within  the  two  settings.  Although  the  experiment 
was  not  modeled  after  a  particular  en  route  center,  the 
scenarios  were  designed  to  reflect  real-world  events. 
Thus,  one  might  expect  similarities  between  intra- 
EST  communications  within  a  field  and  experimental 
setting.  However,  differences  are  also  likely  because 
communications,  as  a  whole,  are  the  product  of  vari¬ 
ous  person  (e.g.,  personality)  and  environmental  fac¬ 
tors  (sector/scenario  demands). 

Table  6  shows  the  comparisons  for  the  percentage 
of  R-side  and  D-side  communications  related  to  the 
topic  of  communication,  its  grammatical  form,  and 
the  mode  of  expression.  Although  Table  6  shows  that 
field  and  laboratory  settings  differed  in  the  percentage 
of  total  communications  that  were  attributed  to  a 
given  topic,  both  laboratory  and  field  assessments 
identified  the  same  top  three  topics.  These  included 
communications  about  “Traffic,”  “Route,”  and  “Alti¬ 
tude.”  Compared  to  the  field  the  most  noticeable 
difference  in  the  experiment  was  the  lack  of  commu¬ 
nications  about  “Weather  (not  part  of  the  experi¬ 
ment),  “Point-Outs,”  and  “Traffic  flow.”  Additionally, 
there  were  only  minimal  amount  of  communications 
concerning  “Flight  Strips.” 

The  grammatical  form  of  communications  also 
differed  between  the  two  environments.  The  field 
results  show  a  strong  tendency  for  the  D-side  to  make 
statements  (55.9%)  and  the  R-side  (42.8%)  to  provide 
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Table  6. 

Contrasting  percentage  comparisons  of  R-side  and  D-side  communications  in  field  and  laboratory 
settings. 


Enroute  Center 

Laboratory  Setting 

R-side% 

D-side% 

R-side% 

D-side% 

Communication  Topic 

Traffic 

41.0 

37.9 

53.7 

51.2 

Route  of  flight 

14.2 

15.6 

13.1 

11.7 

Altitude 

7.1 

8.0 

16.0 

21.1 

Weather 

5.5 

6.8 

0.0 

0.0 

Point-out 

5.0 

6.1 

0.0 

0.0 

Traffic  flow 

5.2 

5.6 

0.0 

0.0 

Frequency 

5.9 

4.7 

3.5 

2.7 

Flight  Strips 

5.6 

4.5 

0.7 

1.0 

Equipment 

3.3 

4.0 

4.5 

4.9 

Hand-off 

3.6 

3.1 

2.9 

1.8 

Speed 

2.6 

2.8 

4.4 

2.8 

Approval 

1.0 

0.9 

1.0 

1.8 

Communication  Format 

Statement 

29.7 

55.9 

58.0 

77.3 

Answer 

42.8 

25.1 

18.3 

10.4 

Question 

12.2 

16.4 

22.9 

11.3 

Command  Answer 

5.8 

0.3 

0.0 

0.0 

Command 

0.5 

2.4 

0.8 

1.0 

Communication  Mode 

Verbal 

77.1 

69.3 

93.9 

69.0 

Verbal  &  Nonverbal 

14.7 

16.8 

5.0 

24.7 

Nonverbal 

13.9 

13.9 

0.5 

2.8 

Equipment 

0.0 

0.1 

Equipment  &  Verbal 

0.6 

3.4 

Equipment  &  Nonverbal 

0.0 

0.1 

9 


answers.  In  contrast,  the  experiment's  results  show 
both  R-side  and  D-side  predominately  making  state¬ 
ments  (58%  vs.  77.3%). 

Verbal  communication  is  the  method  of  choice  for 
R-side  and  D-side  controllers.  However,  as  Table  6 
shows,  the  D-side  had  a  stronger  tendency  to  use  a 
mixture  of  verbal  and  nonverbal  expressions  in  the 
experiment  than  did  the  R-side  (24.7%  vs.  5%).  For 
the  field  setting,  the  percentage  of  mixed  messages  was 
similar  for  both  the  R-side  and  D-side  (14.6%  vs. 
16.8%).  Another  difference  between  the  two  settings 
is  the  percentage  of  nonverbal  communications  that 
were  used.  In  the  field,  13.9%  of  the  communications 
was  solely  nonverbal  for  both  the  R-side  and  D-side. 
This  is  in  contrast  to  the  lower  percentages  recorded 
during  the  experiment  (R-side  0.5%,  D-side  2.8%). 

Hypothesis  Testing 

During  the  experiment,  problems  occurred  with 
the  computer  algorithm  used  to  assign  participants  to 
a  given  condition.  This  created  missing  data  for  three 
teams  in  time  1  and  three  different  teams  in  time  2. 
Because  the  communication  component  of  the  ex¬ 
periment  was  primarily  descriptive  in  nature,  the 
decision  was  reached  not  to  replace  missing  data  with 
mean  substitutions,  but  instead  to  drop  the  cases  from 
further  analysis.  Furthermore,  since  time  1  and  time 
2  data  involved  missing  data  for  six  different  teams, 
the  doubly  repeated  measure  design  was  dropped  and 
changed  to  a  simple  2x3  repeated  measure  design 
using  only  time  1  data.  From  these  data,  3,194  com¬ 
munication  events  were  coded.  Descriptive  statistics 
for  these  data  are  presented  in  Table  7.  These  data  are 
also  graphically  displayed  in  Figures  1-3. 

A  2  x  3  repeated  measures  analysis  of  variance 
(AN OVA)  was  used  to  examine  the  effects  that  air¬ 
craft  density  (low  vs.  high)  and  level  of  technology 
(Techl,  Tech2,  andTech3)  had  on  the  topic  of  intra- 
EST  communication,  its  grammatical  form,  and  the 
method  of  expression. 

Hypothesis  1  stated  that  there  would  be  greater 
amounts  of  communication  under  high  workload 
conditions,  as  compared  to  low  workload  conditions. 
When  communications  were  analyzed  as  a  composite 
number,  no  statistically  significant  results  were  ob¬ 
served.  Each  of  the  C'^T  categories  was  then  analyzed 
separately  to  determine  if  the  experimental  conditions 
affected  any  of  the  subcategories.  No  significant  re¬ 
sults  were  observed  for  any  of  the  subcategories  within 
the  grammatical  form  of  communication  or  the  mode 
of  communication.  However,  main  effects  for 
workload  were  observed  for  two  communication 


topics:  (1)  communications  identifying  a  specific  air¬ 
craft,  F(l,5)  =  11.25.  p  <  .05,  and  communications 
involving  altitude  changes,  F(l,5)  =  10.66,  p  <  .05.  In 
both  cases  the  high  aircraft  density  condition  was 
associated  with  more  communication  exchanges  be¬ 
tween  the  R-side  and  D-side  positions. 

Hypothesis  2  stated  that  more  communications 
would  occur  in  the  higher  tech  condition  as  compared 
to  the  lower  tech  conditions.  Although  statistical 
significance  was  not  attained.  Figure  la  shows  that 
under  the  low  workload  condition,  the  highest  degree 
of  technology  (tech  3)  recorded  the  most  communica¬ 
tion  events  about  Traffic  and  Altitude. 

CONCLUSION 

In  this  study  we  examined  the  effects  of  aircraft 
density  and  different  kinds  of  automated  decision 
aids  on  communication  exchanges  between  R-side 
and  D-side  air  traffic  controller  team  members.  Col¬ 
lapsing  across  all  technologies,  the  results  showed 
more  total  communication  exchanges  under  high  air¬ 
craft  density  conditions  as  compared  to  low  aircraft 
density  conditions.  This  result  was  driven  by  the  top 
two  topics  of  communication:  (1)  communications 
about  the  identification  of  a  specific  aircraft,  and  (2) 
altitude  changes.  The  effects  of  automation  decision 
aids  on  communications,  however,  were  not  as  clear. 
There  was  some  evidence  that  under  the  low  aircraft 
density  condition,  more  communication  exchanges 
occurred  (about  the  identification  of  a  specific  air¬ 
craft,  altitude  changes,  and  route  of  flight)  using  the 
highest  level  of  automation,  as  compared  with  the  two 
lower  levels  of  automation. 

The  overall  trend  of  the  C^T  profile  of  the  experi¬ 
ments  (collapsing  across  all  conditions)  compared 
favorably  with  data  collected  from  the  field.  In  both 
cases,  the  top  three  topics  of  communication  were 
Traffic,  Route,  and  Altitude.  Also,  in  both  cases, 
verbal  communication  was  the  dominant  mode.  De¬ 
spite  these  similarities  there  were  notable  differences 
in  the  grammatical  form  of  communication  between 
R-side  and  D-side  controllers.  Whereas  in  the  field  it 
appeared  that  the  R-side  was  primarily  responding  to 
statements  made  by  the  D-side,  in  the  experiment,  it 
appeared  that  both  R-side  and  D-side  controllers  were 
issuing  statements.  In  addition,  more  nonverbal  com¬ 
munications  were  observed  in  the  field  as  compared  to 
the  laboratory  environment.  Given  that  communica¬ 
tion  is  a  function  of  the  characteristics  of  team  mem¬ 
bers  and  the  situations  that  they  face,  it  is  not  surprising 
that  differences  occurred.  It  remains  for  future  research 
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Table  7. 

Descriptive  Statistics  of  Experimental  Conditions. 


Tech  1 

Tech  2 

Tech  3 

Aircraft 

Density 

Measure 

n 

M 

SD 

n 

M 

SD 

n 

M 

SD 

Lx)w 

Total  Conmiunication 

5 

63.20 

23.27 

5 

62.60 

27.63 

5 

61.00 

27.07 

Topic  of  Communication 

Approval 

1 

1.0 

1 

1.0 

3 

1.33 

0.58 

Handoff 

3 

3.33 

6.43 

2 

5.0 

5.66 

3 

5.0 

4.58 

Pointout 

0 

0 

0 

Traffic 

5 

24.40 

6.43 

5 

26.20 

21.71 

5 

28.60 

14.28 

Altitude 

5 

8.80 

6.53 

5 

8.80 

9.26 

5 

12.40 

5.94 

Route  of  Flight 

5 

7.4 

3.36 

5 

5.80 

4.55 

5 

9.00 

9.80 

Speed 

5 

3.2 

2.68 

5 

4.8 

5.93 

2 

2.50 

.71 

Weather 

0 

0 

0 

Frequency 

5 

3.4 

1.94 

3 

2.67 

0.58 

4 

2.75 

0.96 

Traffic  Flow 

0 

0 

0 

Flight  Strips 

2 

6.0 

4.24 

0 

0 

Equipment 

2 

8.5 

10.60 

5 

5.0 

3.87 

4 

5.0 

2.45 

Aircraft  ID 

0 

0 

0 

Grammatical  Form  of 
Communication 

Question 

5 

9.00 

4.36 

5 

10.20 

3.56 

5 

12.20 

7.16 

Answer 

5 

9.60 

8.99 

5 

8.80 

4.18 

5 

8.80 

5.22 

Statement 

5 

36.20 

11.12 

5 

42.40 

19.63 

5 

39.60 

16.85 

Communication  Mode 

Verbal 

5 

46.40 

18.30 

5 

53.60 

24.09 

5 

51.60 

23.55 

Mixed  Verbal  Nonverbal 

5 

6.8 

7.05 

5 

6.60 

3.51 

5 

7.80 

4.66 

High 

Total  Communication 

5 

69.00 

31.63 

5 

72.60 

5.43 

5 

60.60 

23.07 

Topic  of  Communication 

Approval 

1 

5.00 

2 

3.0 

1.41 

1 

6.00 

Handoff 

2 

2.0 

3 

2.33 

1.52 

2 

1.50 

.71 

Pointout 

1 

1.50 

0 

0 

Traffic 

5 

30.40 

10.55 

5 

35.40 

13.18 

5 

36.40 

13.16 

Altitude 

5 

20.00 

10.86 

5 

13.60 

6.19 

5 

11.40 

5.94 

Route  of  Flight 

5 

8.20 

9.88 

5 

13.60 

10.11 

5 

5.40 

3.78 
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Table?.  (Continued) 

C"*!  Descriptive  Statistics  of  Experimental  Conditions. 


Tech  1 

Tech  2 

Tech  3 

Aircraft 

Density 

Measure 

n 

M 

SD 

n 

M 

SD 

n 

M 

SD 

High 

Speed 

3 

3.67 

1.53 

4 

3.50 

1.00 

4 

4.75 

5.56 

Weather 

0 

0 

0 

Frequency 

4 

3.75 

3.59 

4 

1.75 

0.50 

4 

3.25 

1.26 

Traffic  Flow 

0 

0 

0 

Right  Strips 

2 

4.0 

1.41 

0 

0 

Equipment 

2 

5.5 

6.36 

3 

4.33 

1.53 

3 

6.33 

3.51 

Aircraft  ID 

0 

0 

0 

Grammatical  Form  of 
Communication 

Question 

5 

8.40 

2.97 

5 

13.60 

2.88 

5 

13.40 

5.03 

Answer 

5 

7.60 

6.43 

5 

11.00 

4.95 

5 

9.40 

6.91 

Statement 

5 

51.80 

23.99 

5 

45.80 

6.38 

5 

42.20 

13.03 

Communication  Mode 

Verbal 

5 

52.80 

24.10 

5 

55.20 

8.76 

5 

56.40 

17.90 

Mixed  Verbal  Nonverbal 

5 

13.80 

8.14 

5 

14.60 

7.06 

5 

8.40 

8.14 
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Figure  la. 

Topic  of  Communication  Comparison  of  Three  Decision  Support  Technologies  Under  Low 
Workload  Conditions. 
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Figure  2a. 

Comparison  of  the  Grammatical  Form  of  Communication  for  Three  Decision  Support 
Technologies  Under  Low  Workload  Conditions. 
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Figure  2b. 

Comparison  of  the  Grammatical  Form  of  Communication  for  Three  Decision  Support 
Technologies  Under  High  Workload  Conditions. 
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Figure  3a. 

Comparison  of  the  Mode  of  Communication  for  Three  Decision  Support 
Technologies  Under  Low  Workload  Conditions. 
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Figure  3b. 

Comparison  of  the  Mode  of  Communication  for  Three  Decision  Support 
Technologies  Under  High  Workload  Conditions. 
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to  determine  whether  and  how  intra-  or  inter-team 
variability  affects  system  outcomes  of  safety,  effi¬ 
ciency,  and  effectiveness. 

Future  Directions 

Objective  measures  of  ATC  intra-team  coordination 
have  remained  an  elusive  goal  for  the  FAA.  Despite  the 
number  of  training  initiatives  targeted  at  improving 
ATC  team  performance,  none  of  them  has  empirically 
demonstrated  that  they  lead  to  improvements  in  intra- 
team  coordination  within  the  job  setting. 

Recently,  the  FAA  funded  a  cooperative  research 
program  with  Kansas  State  University  to  develop  a 
method  of  identifying  expert  performance:  the 
Cochran-Weiss-Shanteau  statistic  (CWS)  (Shanteau, 
2001;  Thomas,  2001;  Weiss,  2001).  In  its  current 
form,  CWS  is  a  summative  index  that  differentiates 
between  levels  of  expertise  residing  within  individu¬ 
als.  However,  a  dynamic  measure  of  performance 
expertise  is  in  the  developmental  stages.  By  extending 
this  measure  to  the  team  level  of  analysis,  researchers 
will  accomplish  a  number  of  objectives.  These  in¬ 
clude:  (1)  objectively  classifying  teams  based  on  their 
levels  of  expertise,  (2)  determining  the  role  that  intra¬ 
team  communication  plays  in  developing  team  exper¬ 
tise,  and  (3)  determining  how  levels  of  team  expertise 
relate  to  sector  outcome  measures  such  as  the  average 
amount  of  fuel  burn  or  time  in  sector. 

Whether  intra-EST  communication  research  takes 
advantage  of  advances  in  the  measurement  of  team 
expertise,  it  is  important  to  explore  the  impact  that  intra- 
EST  communications  has  on  system  safety  and  effi¬ 
ciency  outcomes.  To  the  extent  that  a  relationship  exists, 
researchers  will  be  in  a  better  position  to  develop  more 
sophisticated  models  of  ATC  performance. 
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